Minimal Polynomial
Hermitian Matrices
Definition: An n × n n \times n n × n matrix A A A is said to be Hermitian if A = A H = A ∗ A = A^H = A^* A = A H = A ∗ . Its conjugate transpose is equal to itself. If A A A is real, then A = A T A = A^T A = A T .
Theorem: Let A A A be Hermitian. Then ⟨ x , A x ⟩ \langle x,Ax \rangle ⟨ x , A x ⟩ is real for all x ∈ C n x \in \mathbb{C}^n x ∈ C n .
Proof: We will start with properties of inner products.
1- ⟨ x , y ⟩ = ⟨ y , x ⟩ ‾ \large \langle x,y\rangle = \overline{\langle y,x\rangle } ⟨ x , y ⟩ = ⟨ y , x ⟩ .
Then we will substitute A = A ∗ A = A^* A = A ∗
and in the last step we will use,
2- ⟨ x , P y ⟩ = ⟨ P ∗ x , y ⟩ \large \langle x,Py\rangle = \langle P^*x,y\rangle ⟨ x , P y ⟩ = ⟨ P ∗ x , y ⟩ .
Check the properties of inner products here .
⟨ x , A x ⟩ = ⟨ A x , x ⟩ ‾ = ⟨ A ∗ x , x ⟩ ‾ = ⟨ x , A x ⟩ ‾ ■ \large\langle x,Ax\rangle = \overline{\langle Ax,x\rangle } = \overline{\langle A^*x,x\rangle } = \overline{\langle x,Ax\rangle } \ \ \ \ \ \blacksquare ⟨ x , A x ⟩ = ⟨ A x , x ⟩ = ⟨ A ∗ x , x ⟩ = ⟨ x , A x ⟩ ■
Theorem: Let A A A be Hermitian. Then all eigenvalues of A A A are real.
Proof: Let λ \lambda λ be an eigenvalue of A A A and x x x be the corresponding eigenvector. Then A x = λ x Ax = \lambda x A x = λ x . Then we will use the previous theorem.
λ ⟨ x , x ⟩ = ⟨ λ x , x ⟩ = ⟨ A x , x ⟩ = ⟨ x , A x ⟩ ‾ = ⟨ x , λ x ⟩ ‾ = λ ⟨ x , x ⟩ ‾ = λ ‾ ⟨ x , x ⟩ \large\lambda\langle x,x\rangle = \langle \lambda x,x\rangle = \langle Ax,x\rangle = \overline{\langle x,Ax\rangle } = \overline{\langle x,\lambda x\rangle } = \overline{\lambda\langle x,x\rangle } = \overline{\lambda}\langle x,x\rangle λ ⟨ x , x ⟩ = ⟨ λ x , x ⟩ = ⟨ A x , x ⟩ = ⟨ x , A x ⟩ = ⟨ x , λ x ⟩ = λ ⟨ x , x ⟩ = λ ⟨ x , x ⟩
Since ⟨ x , x ⟩ ≠ 0 \langle x,x\rangle \neq 0 ⟨ x , x ⟩ = 0 , we can divide both sides by ⟨ x , x ⟩ \langle x,x\rangle ⟨ x , x ⟩ .
λ = λ ‾ ■ \large\lambda = \overline{\lambda} \ \ \ \ \ \blacksquare λ = λ ■
Theorem: Let A A A be Hermitian. Then all eigenvectors corresponding to distinct eigenvalues are orthogonal. Let A A A be Hermitian and λ i ≠ λ j \lambda_i \neq \lambda_j λ i = λ j be two distinct eigenvalues of A A A with corresponding eigenvectors e i e_i e i and e j e_j e j . Then ⟨ e i , e j ⟩ = 0 \langle e_i,e_j\rangle = 0 ⟨ e i , e j ⟩ = 0 .
Proof: Let A A A be Hermitian and λ i ≠ λ j \lambda_i \neq \lambda_j λ i = λ j be two distinct eigenvalues of A A A with corresponding eigenvectors e i e_i e i and e j e_j e j . Then A e i = λ i e i Ae_i = \lambda_i e_i A e i = λ i e i and A e j = λ j e j Ae_j = \lambda_j e_j A e j = λ j e j . Then we will use the previous theorem.
⟨ e i , A e j ⟩ = ⟨ e i , λ j e j ⟩ = λ j ⟨ e i , e j ⟩ ⟨ e i , A e j ⟩ = ⟨ A ∗ e i , e j ⟩ = λ i ⟨ e i , e j ⟩ \large \langle e_i,Ae_j\rangle = \langle e_i,\lambda_je_j \rangle = \lambda_j \langle e_i,e_j\rangle \\
\large \langle e_i,Ae_j\rangle = \langle A^*e_i,e_j\rangle =\lambda_i \langle e_i,e_j\rangle ⟨ e i , A e j ⟩ = ⟨ e i , λ j e j ⟩ = λ j ⟨ e i , e j ⟩ ⟨ e i , A e j ⟩ = ⟨ A ∗ e i , e j ⟩ = λ i ⟨ e i , e j ⟩
Then we will subtract the equations.
( λ j − λ i ) ⟨ e i , e j ⟩ = 0 \large (\lambda_j - \lambda_i)\langle e_i,e_j\rangle = 0 ( λ j − λ i ) ⟨ e i , e j ⟩ = 0
Since λ j ≠ λ i \large \lambda_j \neq \lambda_i λ j = λ i , we have ⟨ e i , e j ⟩ = 0 ■ \large \langle e_i,e_j\rangle = 0 \ \ \ \ \ \blacksquare ⟨ e i , e j ⟩ = 0 ■
Theorem: Let A A A be Hermitian. Then its minimal polynomial is
m ( s ) = ( s − λ 1 ) ( s − λ 2 ) ⋯ ( s − λ σ ) \large m(s) = (s - \lambda_1 ) (s - \lambda_2 ) \cdots (s - \lambda_{\sigma} ) m ( s ) = ( s − λ 1 ) ( s − λ 2 ) ⋯ ( s − λ σ )
where λ i \lambda_i λ i are the distinct eigenvalues of A A A .
Proof: Set Equality We will prove that m ( s ) m(s) m ( s ) has no repeated roots. For which we need to use N ( ( A − λ I ) ) = N ( ( A − λ I ) 2 ) N((A-\lambda I)) = N((A-\lambda I)^2) N (( A − λ I )) = N (( A − λ I ) 2 ) .
In order to show the equality, we will prove that N ( ( A − λ I ) ) ⊆ N ( ( A − λ I ) 2 ) N((A-\lambda I)) \subseteq N((A-\lambda I)^2) N (( A − λ I )) ⊆ N (( A − λ I ) 2 ) and N ( ( A − λ I ) 2 ) ⊆ N ( ( A − λ I ) ) N((A-\lambda I)^2) \subseteq N((A-\lambda I)) N (( A − λ I ) 2 ) ⊆ N (( A − λ I )) .
1- N ( ( A − λ I ) ) ⊆ N ( ( A − λ I ) 2 ) N((A-\lambda I)) \subseteq N((A-\lambda I)^2) N (( A − λ I )) ⊆ N (( A − λ I ) 2 )
Let x ∈ N ( ( A − λ I ) ) x \in N((A-\lambda I)) x ∈ N (( A − λ I )) . Then ( A − λ I ) x = 0 (A-\lambda I) x = 0 ( A − λ I ) x = 0 . Then we will multiply both sides by ( A − λ I ) (A-\lambda I) ( A − λ I ) .
( A − λ I ) 2 x = 0 (A-\lambda I)^2 x = 0 ( A − λ I ) 2 x = 0
Then x ∈ N ( ( A − λ I ) 2 ) x \in N((A-\lambda I)^2) x ∈ N (( A − λ I ) 2 ) .
2- N ( ( A − λ I ) 2 ) ⊆ N ( ( A − λ I ) ) N((A-\lambda I)^2) \subseteq N((A-\lambda I)) N (( A − λ I ) 2 ) ⊆ N (( A − λ I ))
Let x ∈ N ( ( A − λ I ) 2 ) x \in N((A-\lambda I)^2) x ∈ N (( A − λ I ) 2 ) . Then ( A − λ I ) 2 x = 0 (A-\lambda I)^2 x = 0 ( A − λ I ) 2 x = 0 .
⟨ x , ( A − λ I ) 2 x ⟩ = 0 \langle x, (A-\lambda I)^2 x \rangle = 0 ⟨ x , ( A − λ I ) 2 x ⟩ = 0
⟨ x , ( A − λ I ) ( A − λ I ) x ⟩ = 0 \langle x, (A-\lambda I) (A-\lambda I) x \rangle = 0 ⟨ x , ( A − λ I ) ( A − λ I ) x ⟩ = 0
⟨ ( A − λ I ) x , ( A − λ I ) x ⟩ = 0 = ∥ ( A − λ I ) x ∥ 2 ⟹ ( A − λ I ) x = 0 \langle (A-\lambda I) x, (A-\lambda I) x \rangle = 0 = \|(A-\lambda I)x\|^2 \implies (A-\lambda I)x = 0 ⟨( A − λ I ) x , ( A − λ I ) x ⟩ = 0 = ∥ ( A − λ I ) x ∥ 2 ⟹ ( A − λ I ) x = 0
Then x ∈ N ( ( A − λ I ) ) x \in N((A-\lambda I)) x ∈ N (( A − λ I )) .
Therefore Hermitian matrices A A A with distinct eigenvalues have no repeated roots in their minimal polynomials. ■ \blacksquare ■
d ( s ) = ( s − λ 1 ) ( s − λ 2 ) ⋯ ( s − λ σ ) \large d(s) = (s - \lambda_1 ) (s - \lambda_2 ) \cdots (s - \lambda_{\sigma} ) d ( s ) = ( s − λ 1 ) ( s − λ 2 ) ⋯ ( s − λ σ )
m ( s ) = ( s − λ 1 ) ( s − λ 2 ) ⋯ ( s − λ σ ) \large m(s) = (s - \lambda_1 ) (s - \lambda_2 ) \cdots (s - \lambda_{\sigma} ) m ( s ) = ( s − λ 1 ) ( s − λ 2 ) ⋯ ( s − λ σ )
C n = N ( ( A − λ 1 I ) ) ⊕ N ( ( A − λ 2 I ) ) ⊕ ⋯ ⊕ N ( ( A − λ σ I ) ) \large \mathbb{C}^n = N((A-\lambda_1 I)) \oplus N((A-\lambda_2 I)) \oplus \cdots \oplus N((A-\lambda_{\sigma} I)) C n = N (( A − λ 1 I )) ⊕ N (( A − λ 2 I )) ⊕ ⋯ ⊕ N (( A − λ σ I ))
Theorem: Let A A A be Hermitian matrix with characteristic polynomial d ( s ) = ( s − λ 1 ) r 1 ( s − λ 2 ) r 2 ⋯ ( s − λ σ ) r σ d(s) = (s - \lambda_1 )^{r_1} (s - \lambda_2 )^{r_2} \cdots (s - \lambda_{\sigma} )^{r_{\sigma}} d ( s ) = ( s − λ 1 ) r 1 ( s − λ 2 ) r 2 ⋯ ( s − λ σ ) r σ . Then there exist a unitary matrix P P P such that P − 1 = P ∗ P^{-1} = P^* P − 1 = P ∗ and P ∗ A P = Λ P^*AP = \Lambda P ∗ A P = Λ where Λ \Lambda Λ is a diagonal matrix with diagonal entries λ 1 , λ 2 , ⋯ , λ σ \lambda_1, \lambda_2, \cdots, \lambda_{\sigma} λ 1 , λ 2 , ⋯ , λ σ .
Λ = [ λ 1 0 0 ⋯ 0 0 λ 2 0 ⋯ 0 0 0 λ 3 ⋯ 0 ⋮ ⋮ ⋮ ⋱ ⋮ 0 0 0 ⋯ λ σ ] and each Λ i ∈ C r i × r i , Λ i = [ λ i 0 0 ⋯ 0 0 λ i 0 ⋯ 0 0 0 λ i ⋯ 0 ⋮ ⋮ ⋮ ⋱ ⋮ 0 0 0 ⋯ λ i ] \large \Lambda = \begin{bmatrix} \lambda_1 & 0 & 0 & \cdots & 0 \\ 0 & \lambda_2 & 0 & \cdots & 0 \\ 0 & 0 & \lambda_3 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & \lambda_\sigma \end{bmatrix} \text{ and each }\Lambda_i \in \mathbb{C}^{r_i \times r_i} \text{ , } \Lambda_i = \begin{bmatrix} \lambda_i & 0 & 0 & \cdots & 0 \\ 0 & \lambda_i & 0 & \cdots & 0 \\ 0 & 0 & \lambda_i & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & \lambda_i \end{bmatrix} Λ = ⎣ ⎡ λ 1 0 0 ⋮ 0 0 λ 2 0 ⋮ 0 0 0 λ 3 ⋮ 0 ⋯ ⋯ ⋯ ⋱ ⋯ 0 0 0 ⋮ λ σ ⎦ ⎤ and each Λ i ∈ C r i × r i , Λ i = ⎣ ⎡ λ i 0 0 ⋮ 0 0 λ i 0 ⋮ 0 0 0 λ i ⋮ 0 ⋯ ⋯ ⋯ ⋱ ⋯ 0 0 0 ⋮ λ i ⎦ ⎤
where λ i \lambda_i λ i are the distinct eigenvalues of A A A .
Proof : See the lecture notes, page 50.
Theorem: Let A A A be Hermitian matrix with eigenvalues λ 1 , λ 2 , ⋯ , λ σ \lambda_1, \lambda_2, \cdots, \lambda_{\sigma} λ 1 , λ 2 , ⋯ , λ σ . Let λ m i n = min i = 1 , 2 , ⋯ , σ λ i \lambda_min = \min_{i=1,2,\cdots,\sigma} \lambda_i λ m in = min i = 1 , 2 , ⋯ , σ λ i and λ m a x = max i = 1 , 2 , ⋯ , σ λ i \lambda_max = \max_{i=1,2,\cdots,\sigma} \lambda_i λ m a x = max i = 1 , 2 , ⋯ , σ λ i . Then for all x ∈ C n x \in \mathbb{C}^n x ∈ C n ,
λ m i n ⟨ x , x ⟩ ≤ ⟨ x , A x ⟩ ≤ λ m a x ⟨ x , x ⟩ \large \lambda_{min} \langle x,x \rangle \leq \langle x,Ax \rangle \leq \lambda_{max} \langle x,x \rangle λ min ⟨ x , x ⟩ ≤ ⟨ x , A x ⟩ ≤ λ ma x ⟨ x , x ⟩
Proof: See the lecture notes, page 51.
Definition: A Hermitian matrix A A A is said to be positive definite if ⟨ x , A x ⟩ > 0 \langle x,Ax \rangle > 0 ⟨ x , A x ⟩ > 0 for all x ∈ C n x \in \mathbb{C}^n x ∈ C n and x ≠ 0 x \neq 0 x = 0 . A Hermitian matrix A A A is said to be positive semidefinite if ⟨ x , A x ⟩ ≥ 0 \langle x,Ax \rangle \geq 0 ⟨ x , A x ⟩ ≥ 0 for all x ∈ C n x \in \mathbb{C}^n x ∈ C n .
Proof: By contradiction. Let A A A be positive definite Hermitioan, Then λ i > 0 ∀ i = 1 , 2 , ⋯ , σ \lambda_i > 0 \ \forall i = 1,2,\cdots,\sigma λ i > 0 ∀ i = 1 , 2 , ⋯ , σ .
→ \rightarrow → Suppose not, Let e ≤ 0 e \leq 0 e ≤ 0 be an eigenvector of A A A .
From the positive definiteness of A A A , we have ⟨ e , A e ⟩ > 0 \langle e,Ae \rangle > 0 ⟨ e , A e ⟩ > 0 .
⟨ e , A e ⟩ = ⟨ e , λ e ⟩ = λ ∗ ⟨ e , e ⟩ = λ ⟨ e , e ⟩ > 0 \langle e,Ae \rangle = \langle e,\lambda e \rangle = \lambda^* \langle e,e \rangle =\lambda \langle e,e \rangle > 0 ⟨ e , A e ⟩ = ⟨ e , λ e ⟩ = λ ∗ ⟨ e , e ⟩ = λ ⟨ e , e ⟩ > 0
λ ⟨ e , e ⟩ > 0 ⟹ λ > 0 \lambda \langle e,e \rangle > 0 \implies \lambda > 0 λ ⟨ e , e ⟩ > 0 ⟹ λ > 0 which is a contradiction. ■ \blacksquare ■
#EE501 - Linear Systems Theory at METU